Robust Speech Recognition Based on Multi Stream Features

In this paper we discuss a new automatic speech recognition ASR approach based on the inde pendent processing and recombination of several feature streams In this framework it is assumed that the speech signal is represented in terms of multiple input streams each input stream representing a di erent characteristic of the signal If the streams are entirely synchronous they may be accommodated simply However as discussed in the paper it may be required to permit some degree of asynchrony between streams which are then forced to re combine at some temporal anchor points associated with some pre de ned speech unit levels We start by intro ducing the basic framework of a statistical structure that can accommodate multiple observation streams This ap proach was initially applied to the case of subband based speech recognition and was shown to yield signi cantly bet ter noise robustness After having summarized these res ults the multi stream approach will be used to combine multiple time scale features in ASR systems in our case to use syllable level features in a phoneme based HMM